Adding Android classpath to lint tests

[bst...@slack-corp.com]

When writing tests for my custom lint rules, is there an easy way to pull the Android classpath into the lint project under test instead of stubbing out classes like Activity or Context?

For example, I sometimes get the following error running my test when I include Android classes in my java text under test. Overriding allowCompilationErrors seems to allow Lint Issues with Severity.ERROR to go undetected.

junit.framework.AssertionFailedError: Found compilation problems in lint test not overriding allowCompilationErrors():
DummyProvider.java:Error: 1: The type java.lang.Object cannot be resolved. It is indirectly referenced from required .class files
DummyProvider.java:Error: 2: The import android cannot be resolved
DummyProvider.java:Error: 3: The import android cannot be resolved
DummyProvider.java:Error: 4: ContentProvider cannot be resolved to a type
DummyProvider.java:Error: 6: Context cannot be resolved to a type
DummyProvider.java:Error: 6: The method getContext() is undefined for the type DummyProvider
Example.java:Error: 2: The import android cannot be resolved
Example.java:Error: 3: The import android cannot be resolved
Example.java:Error: 4: The import android cannot be resolved
Example.java:Error: 5: The import android cannot be resolved
Example.java:Error: 7: TextView cannot be resolved to a type
Example.java:Error: 8: Implicit super constructor Object() is undefined. Must explicitly invoke another constructor
Example.java:Error: 8: TextView cannot be resolved to a type
Example.java:Error: 9: TextView cannot be resolved to a type
Example.java:Error: 12: Activity cannot be resolved to a type
Example.java:Error: 12: Activity cannot be resolved to a type
Example.java:Error: 12: TextView cannot be resolved to a type

Or, should I have to stub them out?

My current project setup is such that I have a lint subproject, and apply the lint.jar to the app subproject with this gradle plugin. In my lint subproject gradle file, I have made the jar task dependent on the test task (jar.dependsOn test). This combo makes it so that tests are always run before the jar is built and plugin then places that jar in directory so that the lint task will find it.

The problem is that when I use Android Studio to build a release APK, the tests fail due to the above error. However, building the debug APK from Android Studio, building the debug or release APK from the command line, or running the lint subproject's test suite alone, the tests don't run into the compilation error above and behave as expected.

[Tor Norbye]

When lint's own unit tests are running, they're running in the tools source tree, and it points to its own checked in version of the SDK, so the tests are always pointed to a valid and consistent SDK, so the tests are stable. (By the way, the way lint's tests ensure that they're always using this SDK and not picking up individual developer SDKs from $ANDROID_HOME is that these tests basically all extend the same subclass of LintDetectorTest, where they override the lint() call to also call

        task.sdkHome(sdk);

to point to an SDK install that is found in a relative directory.

Anyway, that obviously doesn't work when lint unit tests are run for third party lint rules outside the lint distribution. What happens right now is that the test infrastructure looks for lint in a couple of places, the main one being consulting $ANDROID_HOME.  Therefore, if you've set $ANDROID_HOME to point to a valid SDK, then the lint unit tests *should* locate the SDK and the android.jar is automatically added to the classpath -- and your tests should be able to refer to android.app.Activity & friends without any compilation/type resolution problems.

Your problem sounds similar to the problem Lin mentioned here the other day where a coworker had problems because they hadn't set $ANDROID_HOME.

I've just added some fixes to the lint test infrastructure (for 2.4 alpha1) which should help with this. (I've only added this to the new test infrastructure, e.g. TestLintTask#lint(), not the older extends LintDetectorTest architecture).

There are two new flags:

* allowMissingSdk: Whether, when lint unit tests are run, the SDK is allowed to be missing. By default, this is false. That means that as of 2.4 if you run a lint unit test and lint can't find an SDK, the test will fail. You can set this to true if for some reason you don't want this behavior but I think this will generally be helpful to default to false: you'll find out if for some reason the SDK isn't there, since otherwise you get strange failures (like the ones Lin mentioned the other day). The test failure also lists how you can go about configuring the SDK. One way is to set $ANDROID_HOME. But that isn't great since it can vary from developer to developer (unless the issue is on a build server where you can set up the environment exactly as you want). Another way is to call TestLintTask#sdkHome(File), as is done by our internal lint unit tests, so you can point to a specific SDK that you want to use for the tests.

* requireCompileSdk: Normally, lint tries to use the SDK that the test's compileSdkVersion asks for, but if it can't find that, it just uses the latest available SDK instead. If you set this flag to true, it will check and make sure that that specific SDK is available, and if not, it will fail the test. This lets you write a test that will fail if it's not using the exact, intended SDK version to compile your test.

Example usages:

lint()

.files(....).

.requireCompileSdk(true)

.run()

.expectClean();

lint()

.files(....).

. allowMissingSdk(true)

.run()

.expect("blah blah");

[bst...@slack-corp.com]

I saw that thread, but had a little trouble following it. Thanks for the clarity and looking forward to 2.4!

Just for a little more context, I have $ANDROID_HOME properly on my machine and the tests only seem to fail when triggered by the "Build APK" task in Android Studio 2.3 (and 2.2). I even added the $ANDROID_HOME as a path variable in the Android Studio settings.

[Tor Norbye]

Can you try System.getenv("ANDROID_HOME") from within your test to see if that environment variable is somehow not making it through? (I vaguely remember that there are some challenges passing environment variables and system properties to gradle build scripts this way - the daemon is started early on via a tooling api, not in a straightforward Runtime.getDefault().exec-way where you can pass an environment list.)

On Thu, Mar 9, 2017 at 11:27 AM <bst...@slack-corp.com> wrote:

I saw that thread, but had a little trouble following it. Thanks for the clarity and looking forward to 2.4!

Just for a little more context, I have $ANDROID_HOME properly on my machine and the tests only seem to fail when triggered by the "Build APK" task in Android Studio 2.3 (and 2.2). I even added the $ANDROID_HOME as a path variable in the Android Studio settings.

--
You received this message because you are subscribed to the Google Groups "lint-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lint-dev+u...@googlegroups.com.
To post to this group, send email to lint...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lint-dev/25041c38-34f1-460b-ac1a-8cbfd59b2c76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ilya Zorin]

Hi Tor,

I have some weird issues related to $ANDROID_HOME in my project. Basically, the same project behaves differently on different machines. On the first one, everything works fine. On another one, I am not able to run lint related tests from Android Studio due to java.lang.AssertionError: This test requires an Android SDK: No SDK configured, but the same tests work if I run them from a command line. Both of machines have $ANDROID HOME set up properly. I tried System.getenv("ANDROID_HOME") and it gives me null in non-working case. I am on Android Studio RC1 and RC1 lint dependencies as well. Am I missing something? Thanks in advance!

[Tor Norbye]

On Thursday, October 19, 2017 at 5:51:09 AM UTC-7, Ilya Zorin wrote:

Hi Tor,

I have some weird issues related to $ANDROID_HOME in my project. Basically, the same project behaves differently on different machines. On the first one, everything works fine. On another one, I am not able to run lint related tests from Android Studio due to java.lang.AssertionError: This test requires an Android SDK: No SDK configured, but the same tests work if I run them from a command line. Both of machines have $ANDROID HOME set up properly. I tried System.getenv("ANDROID_HOME") and it gives me null in non-working case. I am on Android Studio RC1 and RC1 lint dependencies as well. Am I missing something? Thanks in advance!

The way we do this for the builtin lint checks, to ensure that we're always using the exact same SDK for all developers, on build servers, etc -- is that instead of extending LintDetectorTest, all the lint tests are extending a subclass of LintDetectorTest which does this:

static File sdk;
static {
    sdk = TestUtils.getSdk();
}

@Override
@NonNull
protected TestLintTask lint() {
    TestLintTask task = TestLintTask.lint();
    task.sdkHome(sdk);
    return task;
}

TestUtils.getSdk() is a utility method which produces a path to a full SDK install which we've checked into version control.

The purpose of getSdk() 

Then we're replacing the lint() command used to start lint tests with one where we preconfigure the sdk home to point to our specific SDK.

-- Tor