Lint not running on test sources (src/test)

[Vojtěch Pešek]

Hi lint developers,

currently I've been trying to implement a custom lint rule. Unfortunately I haven't been able to get it run on Test sources (located at src/test). Since our Android Project is large and have lots of gradle "oddities" I've tried moving my custom lint rule to a new Project. Unfortunately, I've got no success there either.

From looking at threads [1] [2] in the lint-dev forum, it should work, yet I was unable to make it work.

The Implementation of the issue contains

EnumSet.of(Scope.JAVA_FILE, Scope.TEST_SOURCES)

and to verify, that my Detector actually works I wrote few unit tests and they are passing. Running the gradle task: ./gradlew :app:lintDebug yields no errors, altought it shoulld. Here is public repository, where you can check the files out. There is app module and lint-rules module. First module contains 2 unit test (java and kotlin), which I would expect the lint to report. The lint-rules module contains IssueRegistry, Detector and respective unit tests.

There are some warnings about the lint registry from kotlin libs, but that shouldn't be a problem, right?

Custom lint rule jar /Users/pesek/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.4.31/6dd50665802f54ba9bc3f70ecb20227d1bc81323/kotlin-stdlib-common-1.4.31.jar does not contain a valid registry manifest key (Lint-Registry-v2).
Either the custom jar is invalid, or it uses an outdated API not supported this lint client

I'm not sure if the setup is not right or I missunderstood something about the scopes. If you could look into it, that would be great.

Best regards,

Vojtech P.

[Tor Norbye]

The problem is that you look up annotations and compare them by name, but you're not comparing by qualified name. The following patch should make it work.

-            if (ann.contains(Pair("RunWith", "RobolectricTestRunner")) && !ann.contains(Pair("Category", "RobolectricTest"))) {

+            if (ann.contains(Pair("org.junit.runner.RunWith", "org.robolectric.RobolectricTestRunner")) && ...

The reason this worked in your test is that you're referencing classes there that you haven't provided as stubs.

See https://googlesamples.github.io/android-custom-lint-rules/api-guide.html#lintcheckunittesting/librarydependenciesandstubs 

-- Tor

[Vojtěch Pešek]

Hi Tor,

Thanks for the quick response. You were right, the important thing I didn't previously understood was how to stub properly and without stubbing, there is difference between name and qualified name.

I did some small changes for kotlin annotations and both work now - highlights in the IDE and our CI gradle task.

Best regards,

Vojtech

Dne sobota 29. května 2021 v 2:08:59 UTC+2 uživatel Tor Norbye napsal:

[Tor Norbye]

This has been a source of confusion for a long time.

But we can actually do a bit better -- so we've just integrated a test framework feature where lint will go and verify that in each test case, it *can* resolve any symbol which is referenced in an import, as well as any call or reference that matches methods and references that a detector has registered interest in (via getApplicableMethodNames() and getApplicableReferenceNames()).

Here's some examples of what you'll see for unresolvable symbols, method calls and references:

https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:lint/libs/lint-tests/src/test/java/com/android/tools/lint/checks/infrastructure/ResolveCheckerTest.kt

Note that this means that after upgrading to the new version of the test library, it's possible that some of your tests will start failing. But this is probably a good thing, because you might not have been testing everything you thought.

-- Tor