The reason that warning tells you that it's a Gradle project and to use gradle instead is precisely because of this. Lint can't interpret the Groovy or Kotlin files in Gradle's build files, as well as any logic performed by third party Gradle plugins applied by the script, which can do things like add new source folders into project. If it's a Gradle project, you really should run Gradle's integration of lint which uses Gradle to figure out where the sources are, where the bytecode outputs are, what language levels to apply, and more.
If you remove the build.gradle files to trick it, it will assume that the directory layout is one of the old Android project layouts (which had AndroidManifest.xml at the root, Java sources in src/ and resources in res/ and will do some simple searches. It will not find src/main/java, src/test/java, etc. There are additional flags (see lint --help) to point it to other locations for manifest files, java source files, class files, etc. But that will flatten the world; lint won't know metadata like whether a manifest is in a library file, etc.