How to unit test a detector that is only in Scope.TEST_SOURCES

[Howard Liberty]

I am trying to write a Detector that works only for tests not app code.

Issue ISSUE = Issue.create(id,description,detail_description, Category.CORRECTNESS, Severity.ERROR, new Implementation(MyDetector.class, EnumSet.of(Scope.TEST_SOURCES)));

The unit test is like this

lint().files(java(SOURCE_CODE).issues(MyDetector.ISSUE).run().expect(expectedError);

I found the the detector is not running for the specific source. If I change the scope to Scope.JAVA_FILE, it works fine. However, I want to make the specific lint check for test sources only.

Is there a way to unit test that? Thanks!

[Tor Norbye]

The way lint's unit testing infrastructure decides whether source file is in a test source set or a production source set is based on the initial directory; if it's in "src/" it's a regular source and if it's in "test/" it's a test source. (In a Gradle test project it would be src/main/java/ and src/main/test/ instead).

Example:

lint().files(
    java( ... ),
    // test/ prefix makes it a test folder entry:
    java(
        "test/my/pkg/UnitTest.java", 
        """
        package my.pkg;
        
        public class UnitTest {
            public void test() {
                new ProductionCode().code(); // OK
                new ProductionCode().testHelper1(); // OK
                new ProductionCode().testHelper2(); // OK
                
            }
        }
        """
    ).indented(),
    java( ... )
).run().expect(expected)

Normally you don't need to specify the path for a test file class since it will usually just infer it from the package declaration and class name in the source, so you can leave out that first argument, but for test sources you'd include it like the above.

-- Tor

[Howard Liberty]

Thanks a lot Tor. Is it possible to target the lint rule to both test/ and androidTest/ folder? I found my lint rule does not seem to run in androidTest folder if I specify the scope TEST_SOURCES.

[Tor Norbye]

The unit testing infrastructure probably needs to be updated to handle androidTest/ as a prefix; can you file a request for that?

-- Tor

On Thu, Aug 2, 2018 at 8:23 AM Howard Liberty <mrlhwl...@gmail.com> wrote:

Thanks a lot Tor. Is it possible to target the lint rule to both test/ and androidTest/ folder? I found my lint rule does not seem to run in androidTest folder if I specify the scope TEST_SOURCES.

--
You received this message because you are subscribed to the Google Groups "lint-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lint-dev+u...@googlegroups.com.
To post to this group, send email to lint...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lint-dev/e5eda994-822c-4290-96b7-892184f16a60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tor Norbye]

Oh wait, are you saying that your lint check itself, with the scope TEST_SOURCES, does not seem to run in instrumentation test sources when running from Gradle or the IDE; e.g. it's a bug in the way TEST_SOURCES is handled, unrelated to the lint unit testing infrastructure?

On Thu, Aug 2, 2018 at 8:23 AM Howard Liberty <mrlhwl...@gmail.com> wrote:

Thanks a lot Tor. Is it possible to target the lint rule to both test/ and androidTest/ folder? I found my lint rule does not seem to run in androidTest folder if I specify the scope TEST_SOURCES.

[Howard Liberty]

Ok after some more testing, I found androidTest/ works if I add Scope.JAVA_FILES. I can summarize the two issues here:

1. Unit test lint rule with test source: SOLVED by specifying the source path as suggested.

2. Lint rule can target to both source and test source by specifying both java files and test sources. This works for src/ test/ and androidTest/

new Implementation(MyDetector.class, EnumSet.of(Scope.JAVA_FILES, Scope.TEST_SOURCES));

However if I just want to target test/ and androidTest/, it is impossible to do by. The lint rule does not run on either test/ or androidTest/

new Implementation(MyDetector.class, EnumSet.of(Scope.TEST_SOURCES))

From the source code of LintDriver.kt, it seems like just linting test sources is not supported. The logic adds test sources only if the scope has JAVA_FILES or ALL_JAVA_FILES.

https://android.googlesource.com/platform/tools/base/+/studio-master-dev/lint/libs/lint-api/src/main/java/com/android/tools/lint/client/api/LintDriver.kt#953

Tor, could you clarify if the use case to apply lint rule to test only code is supported? Thanks!

On Wednesday, August 1, 2018 at 2:36:54 PM UTC-7, Howard Liberty wrote:

[Howard Liberty]

I resolved the issue this way.

First target the lint rule to both java files and test sources

new Implementation(MyDetector.class, EnumSet.of(Scope.JAVA_FILES, Scope.TEST_SOURCES));

Then inside the lint rule I used 

public  UElement  createUastHandler(@NonNull  JavaContext  context){
     return  new  UElementHandler()  {
             @Override
             public void visitXXXX(UExpression node)  {
                   if  (context.isTestSource())  {
                         context.report(XXXX);
                   }
             }
     }
}

This way works to target only the test sources.

However, this is not the most efficient way since it still runs for the non-test sources.

On Wednesday, August 1, 2018 at 2:36:54 PM UTC-7, Howard Liberty wrote:

[Tor Norbye]

Yes, there isn't a way to write a check that looks *only* at test sources. Right now the TEST_SOURCES scope is saying "I *also* apply to tests!" not "I only apply to tests".  Filtering by context.isTestSource is the best way to do what you want right now; we could consider adding a separate scope for this, but it hasn't come up before so I suspect it's not a very common scenario.

-- Tor