Different Results with ClassGraph than with FastClasspathScanner

[John Emmer]

When I was using the following Scala code with FastClasspathScanner 2.9.4, it found 906 "FileWIthAPI" results.

package com.cj.apifinder

import java.lang.reflect.Executable

import com.cj.apifinder.SpecFinder.FileWithApi
import io.github.lukehutch.fastclasspathscanner.FastClasspathScanner
import io.github.lukehutch.fastclasspathscanner.matchprocessor.MethodAnnotationMatchProcessor
import org.springframework.web.bind.annotation.RequestMapping

import scala.collection.mutable
import scala.collection.mutable.ListBuffer

object AnnotationApiFinder {

  def main(args: Array[String]): Unit = {

    findApisViaAnnotations().foreach( println(_) )
  }
 
  def findApisViaAnnotations(): Seq[FileWithApi] = {
    val results = mutable.ListBuffer[FileWithApi]()
    val scanner = new FastClasspathScanner("cj","com.cj")
          .matchClassesWithMethodAnnotation(classOf[RequestMapping], new RequestMappingAnnotationMatchProcessor(results))
          .scan()
    results
  }
}

class RequestMappingAnnotationMatchProcessor(filesWithApiUrls: ListBuffer[FileWithApi]) extends MethodAnnotationMatchProcessor {
  override def processMatch(aClass: Class[_], executable: Executable): Unit = {

    val requestMapping : RequestMapping = executable.getAnnotation(classOf[RequestMapping])
    if ( requestMapping != null )
      filesWithApiUrls ++=  requestMapping.value.map( apiUrl => FileWithApi(aClass.getCanonicalName, Option(apiUrl)) )
    else
      println( aClass.getCanonicalName + " : Why no RequestMapping?" )
  }
}

Now I'm using the following code with ClassGraph 4.0.6 to scan the same classpath, and I only get 58, even though now I'm looking for both class and method annotations:

package com.cj.apifinder

import com.cj.apifinder.SpecFinder.FileWithApi
import io.github.classgraph.AnnotationInfoList.AnnotationInfoFilter
import io.github.classgraph.{AnnotationInfo, AnnotationParameterValue, ClassGraph, ClassInfoList}
import scala.collection.JavaConverters._

object RequestMappingFinder {

  def main(args: Array[String]): Unit = {

    findApisViaAnnotations().foreach( println(_) )
  }

  def findApisViaAnnotations(): Seq[FileWithApi] = {
    val scanResult = new ClassGraph()
          .whitelistPackages("cj","com.cj")
          .enableAllInfo()
          .scan()

    apisIn(scanResult.getClassesWithAnnotation("org.springframework.web.bind.annotation.RequestMapping"))
      .union(apisIn(scanResult.getClassesWithMethodAnnotation("org.springframework.web.bind.annotation.RequestMapping")))
  }

  def apisIn(annotatedClasses: ClassInfoList): Seq[FileWithApi] = {
    annotatedClasses.getNames.asScala
      .flatMap(className => {
        annotatedClasses.get(className)
          .getAnnotationInfo
          .filter(new AnnotationInfoFilter {
            override def accept(annotationInfo: AnnotationInfo): Boolean = {
              return annotationInfo.getName.equals("org.springframework.web.bind.annotation.RequestMapping")
            }
          })
          .asScala.map(annotationInfo => {
            val annotationValueHolder = annotationInfo.getParameterValues.asScala
                  .find({ paramValue: AnnotationParameterValue => paramValue.getName.equals("value") })
            val annotationValue = if (annotationValueHolder.isEmpty) None
                                  else Option(convertValue(annotationValueHolder.get.getValue))
            new FileWithApi(className, annotationValue)
          })
      })
  }

  def convertValue(value: Object ): String = {
    if ( value.isInstanceOf[Array[Object]] ) {
      value.asInstanceOf[Array[Object]](0).toString
    } else {
      value.toString
    }
  }

}

I'm sure I"m just making some silly mistake interpreting the results of the scan - can anyone help me spot it?

Thanks!

- John

[Luke Hutchison]

Interesting... Can you please take verbose logs in both cases, and post Pastebin links here for the logs? 

--
You received this message because you are subscribed to the Google Groups "ClassGraph-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to classgraph-use...@googlegroups.com.
To post to this group, send email to classgra...@googlegroups.com.
Visit this group at https://groups.google.com/group/classgraph-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/classgraph-users/26c8443a-db96-4575-9c2d-f8b1fc43d262%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luke Hutchison]

I took a closer look at this, and I think I see what's going on. You're setting annotatedClasses to the union of classes with class annotations and classes with method annotations of the same annotation type:

    apisIn(scanResult.getClassesWithAnnotation("org.springframework.web.bind.annotation.RequestMapping"))
      .union(apisIn(scanResult.getClassesWithMethodAnnotation("org.springframework.web.bind.annotation.RequestMapping")))


But then your first filter function only iterates through the class annotations. 

    annotatedClasses.getNames.asScala
      .flatMap(className => {
        annotatedClasses.get(className)
          .getAnnotationInfo
          .filter(new AnnotationInfoFilter {
            override def accept(annotationInfo: AnnotationInfo): Boolean = {
              return annotationInfo.getName.equals("org.springframework.web.bind.annotation.RequestMapping")
            }
          })


You need to also call, for each ClassInfo object, ClassInfo#getMethodInfo(), then for each MethodInfo object, MethodInfo#getAnnotationInfo(), then you need to pull out the matching method annotations with non-null values (in other words, you need to match ClassInfo objects with either class or method annotations, but in the filter, you are only matching classes with the correct class annotation). Let me know if this fixes it for you.

By the way, the second code snippet above is inefficient -- you get each class name, then you look the class name up in annotatedClasses again by name. It's better just to iterate through the ClassInfo objects in annotatedClasses, and not call .getNames.

[Luke Hutchison]

P.S. Scala 2.12+ can convert Scala lambdas to Java FunctionalInterfaces, so you can use Scala lambdas for the filters, since the filter classes (AnnotationInfoFilter etc.) are Single Abstract Method interfaces.

[John Emmer]

Aha - thank you!  Yes, I figured I just wasn't understanding how to use the API.  I'll probably do some refactoring, but for now I've updated the code as follows, and it seems to be working as expected: 

  def findApisViaAnnotations(): Seq[FileWithApi] = {
    val scanResult = new ClassGraph()
          .whitelistPackages("cj","com.cj")
          .enableAllInfo()
          .scan()

    apisDeclaredAtClassLevel(scanResult.getClassesWithAnnotation("org.springframework.web.bind.annotation.RequestMapping"))
      .union(apisDeclaredAtMethodLevel(scanResult.getClassesWithMethodAnnotation("org.springframework.web.bind.annotation.RequestMapping")))
  }

  def apisDeclaredAtClassLevel(annotatedClasses: ClassInfoList): Seq[FileWithApi] = {
    annotatedClasses.asScala
      .flatMap(classInfo => {
          classInfo.getAnnotationInfo
          .filter(new AnnotationInfoFilter {

            override def accept(annotationInfo: AnnotationInfo): Boolean = {
              return annotationInfo.getName.equals("org.springframework.web.bind.annotation.RequestMapping")
            }
          })

          .asScala.map(annotationInfo => {
            val annotationValueHolder = annotationInfo.getParameterValues.asScala
                  .find({ paramValue: AnnotationParameterValue => paramValue.getName.equals("value") })
            val annotationValue = if (annotationValueHolder.isEmpty) None
                                  else Option(convertValue(annotationValueHolder.get.getValue))

            new FileWithApi(classInfo.getName, annotationValue)
          })
      })
  }

  def apisDeclaredAtMethodLevel(annotatedClasses: ClassInfoList): Seq[FileWithApi] = {
    annotatedClasses.asScala
      .flatMap(classInfo => {
          classInfo.getMethodInfo.asScala
            .flatMap( methodInfo => methodInfo.getAnnotationInfo
                .filter(new AnnotationInfoFilter {

                  override def accept(annotationInfo: AnnotationInfo): Boolean = {
                    return annotationInfo.getName.equals("org.springframework.web.bind.annotation.RequestMapping")
                  }
                })

                .asScala.map(annotationInfo => {
                  val annotationValueHolder = annotationInfo.getParameterValues.asScala
                        .find({ paramValue: AnnotationParameterValue => paramValue.getName.equals("value") })
                  val annotationValue = if (annotationValueHolder.isEmpty) None
                                        else Option(convertValue(annotationValueHolder.get.getValue))

                  new FileWithApi(classInfo.getName, annotationValue)
                }))
      })
  }

[Luke Hutchison]

Looks right to me. Do I need to add any specific clarifications to the docs that would have made it easier for you to have figured this out?

--
You received this message because you are subscribed to the Google Groups "ClassGraph-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to classgraph-use...@googlegroups.com.
To post to this group, send email to classgra...@googlegroups.com.
Visit this group at https://groups.google.com/group/classgraph-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/classgraph-users/8a339feb-c064-497d-b9fa-217351dc097f%40googlegroups.com.

[John Emmer]

On Monday, August 20, 2018 at 12:16:47 PM UTC-7, Luke Hutchison wrote:

Looks right to me. Do I need to add any specific clarifications to the docs that would have made it easier for you to have figured this out?

I think what threw me off was the name 'ClassInfoList', which led me to expect everything to be arranged by class, so even when I was searching for method annotations I thought they would show up in that 'List' by class.

[Luke Hutchison]

I see -- I added info to https://github.com/classgraph/classgraph/wiki/ClassGraph-API#general-usage-pattern specifically on reading method and field annotations (this was a glaring omission). Thanks, and I'm glad it's working for you now!

--

You received this message because you are subscribed to the Google Groups "ClassGraph-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to classgraph-use...@googlegroups.com.
To post to this group, send email to classgra...@googlegroups.com.
Visit this group at https://groups.google.com/group/classgraph-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/classgraph-users/702ba9ee-a474-4dc8-a2e7-42803d4c7f19%40googlegroups.com.